System and method for analyzing, organizing, and presenting data stored on a mobile communication device

ABSTRACT

A system and method is provided for receiving, analyzing, organizing and presenting data stored on a mobile communication device. In one embodiment of the present invention, the data is organized by application-type and is searchable. In one embodiment, the system may be configured to generate first and second search results from a first search term entered by a reviewer, where the first search result includes at least a first record from a first application, the second search result includes at least a second record from a second application, and both records are displayed to the reviewer at the same time. In performing the search, the first record should match the first search term and the second record should match a second search term, where the second is identified by the system and is related to but different from the first search term.

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 16/826,149, filed Mar. 20, 2020, which is acontinuation of U.S. patent application Ser. No. 16/534,197 filed Aug.7, 2019. All extrinsic materials identified in this application areincorporated by reference in their entirety.

FIELD OF THE INVENTION

The present invention relates to eDiscovery, and more particular to asystem and method for receiving, analyzing, and organizing data storedon a mobile communication device (e.g., text messages, phone logs,photos, social media data, etc.) and presenting that data in auser-friendly fashion. For example, the data should be searchable andindividual records should be presented in context, which may involvedisplaying related records within a single application and amongstdifferent applications.

BACKGROUND

The background description includes information that may be useful inunderstanding the present invention. It is not an admission that any ofthe information provided in this application is prior art or relevant tothe presently claimed invention, or that any publication specifically orimplicitly referenced is prior art.

As more and more people are using mobile phones to communicate bothprofessionally and personally, Mobile Phone Discovery (MPD) is anindustry on the rise. By way of example, it was estimated that in 2018,over $10B were spent on eDiscovery software and services, and by theyear 2023, this number is expected to exceed $23B. This increase is dueto several factors, including costs and expenses for developing newtechnology, marketplace expansion, and the number of communicationdevices in use.

For example, with respect to marketplace expansion, the #metoo movement,employment disputes, investigations (HR, law enforcement, etc.),compliance monitoring, and responses to litigation are causing adramatic increase in the need for eDiscovery. eDiscovery is essentiallysoftware that can be used to receive, analyze, organize, and presentdata stored on a mobile phone. Examples of such data include messagingdata (e.g., text messaging), telephone data, photo data, data for socialmedia applications (e.g., Twitter, Instagram, Facebook, WhatsApp, etc.),etc.

Not only is the marketplace expanding, but so too are the number ofmobile communication devices. There is a steep increase in the number ofmobile phones that are being used, as well as other mobile communicationdevices, such as tablets, laptops, etc.

While software for processing data stored on a personal computer (PC)can be used with mobile devices, the results are less than desirable.This is because traditional discovery software treats each file (e.g.,an email, a Word document) as a stand-alone record, whereas on mobiledevices, records are much more interrelated. For example, a text messagedoes not exist in a vacuum but is instead part of a much larger record,generally referred to as a “thread.” Similarly, when one is using asocial media application, such as Facebook or Twitter, individualcommunications are part of the user's Page or account. In other words,unlike a PC, files on mobile devices are interrelated. When viewedtogether, they can provide a nice “snapshot” of the user's activities.

Thus, there is a need for a system and method that addresses the issuesand problems with traditional discovery software. In particular, thereis a need for software that can receive, analyze, organize, and presentdata that is traditionally stored on mobile communication devices, suchas text messages, social media communications, etc. There is also a needto present such data in a user-friendly manner, allowing the reviewer toeasily navigate, easily annotate (e.g., tag, etc.), and provide contextto individual records. It would also be beneficial to use artificialintelligence (AI) to “tell a story,” which may involve displaying datathat is related, both within an application and amongst differentapplications.

SUMMARY OF THE INVENTION

The present invention is directed toward the receiving, analyzing,organizing, and displaying of data stored on a mobile communicationdevice (e.g., text messages, phone logs, photos, social media data,etc.). Preferred embodiments of the present invention operate inaccordance with a Web host in communication with a mobile communicationdevice, such as a mobile phone, via a wide area network (WAN). In oneembodiment of the present invention, a local computer (e.g., a personalcomputer) is connected to the mobile device and configured to receivethe data stored on the mobile device. This data is then transferred tothe Web host, either directly (e.g., by providing the data directly tothe Web host) or indirectly (e.g., by providing the data to the Web hostvia a networked computer, such as a server). In other embodiments, Webhost software (or a portion thereof) is loaded onto the local computer,allowing the data to be analyzed, organized, and presented on the localcomputer.

Regardless of whether the present invention is operating locally orremotely, the reviewer will have an opportunity to navigate, interactwith, and organize data that was previously stored on the mobilecommunication device. This may involve known navigation techniques, suchas browsing and/or searching.

Data (or individual records) on mobile devices are generally associatedwith either the operating system or applications that are operating onthe mobile device. As such, the system may first identify applicationsthat are operating on (or accessible via) the mobile device. Suchapplications may include, for example, a messaging application, a phoneapplication, an email application, a photo application, and otherthird-party applications (e.g., Instagram™ Facebook™, Twitter™,WhatsApp™, etc.). Once the applications are identified, the data (orindividual portions thereof) (e.g., files, records, etc.) can then beassociated (or linked) with at least one application.

Once the individual records are organized (e.g., associated with anapplication), the user can then browse by selecting an application ofinterest. For example, if the phone application is selected, then thereviewer may be presented with a phone log, which may provideinformation on each call, such as incoming, outgoing, answered,rejected, name, number, date, time, duration, etc. If the messageapplication is selected, then the reviewer may be presented with messagedata, which may include individual threads having individual records ormessages. The reviewer may also be provided with information on eachrecord, including, but not limited to, the record's content, to, from,date, time, read, unread, etc. If the photo application is selected,then the reviewer may be presented with photo data, which may includeindividual photos, along with information on each photo (e.g., filesize, location, date, time, whether the photo was edited, whether thephoto was shared, and if so, to whom and when, etc.). Finally, if asocial media application is selected, then the reviewer may be presentedwith data for that application, which may include photos, messages, orother data created, received, or shared using the application.

In one embodiment of the present invention, data is not presented to theuser in isolation, but in context. In other words, when analyzing datastored on a personal computer, reviewing a Word document along with itsmetadata may be sufficient to understand the document, including itscontent, when it was created, when it was edited, etc. However, a textmessage may not make any sense until the message is viewed in context,which may involve reviewing preceding messages, successive messages,messages involving the same participates on other applications, etc. Forexample, Don, Mary, and Paul may be texting each other while Don andMary are also instant messaging each other on Facebook™. Unliketraditional records on a personal computer, these mobile records mayonly make sense when viewed together, preferably at the same time (e.g.,side-by-side, etc.). As discussed in greater detail below, this mayinvolve identifying related records, which may involve identifying andanalyzing identifiers for each record (e.g., a record ID, a thread ID,etc.), associated data (e.g., data that links multiple terms together),etc.

When reviewing the data, the reviewer may be allowed to annotate eachrecord by selecting at last one “tag” from a plurality of tags, whereeach tag has a separate meaning. For example, a tag could show whether afile was reviewed, not reviewed, responsive to a discovery request,non-responsive, responsive to a particular topic (e.g., a particulardiscovery request, a particular investigative issue, etc.), privileged(e.g., attorney-client, work product, etc.), non-privileged, etc. Inaddition, or alternatively, tags can be automatically selected by thesystem (e.g., reviewed, not reviewed, presented for review, reviewer'sname, review date, review time, review device, etc.). Each tag can beassociated with a record or a plurality of records (e.g., a thread,etc.). Once a tag has been selected, it will then be associated with therecord for which it was selected.

As previously discussed, a reviewer may also search the data for atleast one search term (e.g., name, subject, date, etc.). Results mayinclude records that match or are closely related (e.g., similar,abbreviations, etc.) and may be performed global (i.e., on all the data)or on data for individual applications. For example, if a reviewerenters the name “Mary” in a search field, then the system may providesearch results, which may be from one application or many applications.

As discussed above, the present invention may also be configured topresent records (such as search results) in context. This may be doneeither automatically or selectively (e.g., by selecting the “storytelling” feature). For example, if a message is responsive to a searchterm, then the system may present the reviewer with at least onepreceding message and at least one succeeding message from the sameapplication. The system may also present the reviewer with at least onerelated record from another application. This may include a recordinvolving the same person, involving the same or similar subject matter,was received/sent immediately before, immediately after, etc.

For example, a search result for “Mary and store” may return a messagefrom Mary asking Tom whether he's ready to go the hardware store. Inorder to put the message in context, this message may be displayedtogether with a preceding message (e.g., from Tom asking Mary whethershe's ready to go) and a succeeding message (e.g., from Tom stating thathe'll meet her at Home Depot at 3:00). The message may also be displayedtogether with other records, from other applications, such as othermessages between Tom and Mary, other messages mentioning Mary, Webbrowser activity around the time the message was received, Uber™activity around the time the message was received, etc. Data like thismay help the reviewer understand the search results by putting them incontext.

In order to find related records (e.g., tell a story), it may benecessary to build a database, linking data from different applicationstogether. For example, assume that system is searching for “Ardy.” Thesystem may identify text messages that are to or from Ardy and socialmedia involving Ardy. However, the system may also be configured tocollect and organize information about Ardy, which may be extracted fromvarious applications and/or records. For example, a contact record onArdy may include his phone number, his email address, and his mailingaddress, and other records stored on the mobile device may include (orshow) other identifiers, such as his date of birth, government issuedID, financial information, etc. While processing the data stored on themobile device, the system may also learn that some of these identifiersare also linked to a user by the name of A_Akbar on WhatsApp, and to auser by the name Camp_Clown on Facebook. This collection of information(e.g., in a database) can then be used to identify related records. Forexample, a system search for “Ardy” may also pull up records for A_Akbarand Camp_Clown.

A more complete understanding of a system and method for receiving,analyzing, organizing, and presenting data stored on a mobilecommunication device, and presenting that data in a user-friendlyfashion, will be afforded to those skilled in the art, as well as arealization of additional advantages and objects thereof, by aconsideration of the following detailed description of the preferredembodiment. Reference will be made to the appended sheets of drawings,which will first be described briefly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a Web host in communication with a mobilecommunication device, either directly via other devices (e.g., a PC, acloud service, etc.), via a wide area network (WAN), allowing the Webhost to receive data stored on the mobile communication device;

FIG. 2 illustrates one embodiment of the Web host depicted in FIG. 1 ;

FIGS. 3A and B provide exemplary screen shots in accordance with oneembodiment of the present invention, showing that different cases mayinclude different devices having different data that can be reviewed(e.g., via browsing, searching, etc.);

FIG. 4A provides an exemplary screen shot of phone data in accordancewith one embodiment of the present invention;

FIG. 4B shows a database in accordance with one embodiment of thepresent invention, illustrating how tags can be linked to (or associatedwith) at least one file;

FIG. 5 provides an exemplary screen shot of message data in accordancewith one embodiment of the present invention;

FIG. 6 provides an exemplary screen shot of photo data in accordancewith one embodiment of the present invention;

FIG. 7 provides an exemplary screen shot of social media data (e.g.,WhatsApp) in accordance with one embodiment of the present invention;

FIG. 8 shows a database in accordance with one embodiment of the presentinvention, illustrating how various fields (e.g., name, phone, email,etc.) can be used to identify an individual and to link different usernames together;

FIGS. 9-12 provide exemplary screen shots of various search results inaccordance with various embodiments of the present invention;

FIGS. 13 and 14 provide exemplary screen shots for setting up an accountwith a host device or host software operating on a local device inaccordance with various embodiments of the present invention;

FIG. 15 provides an exemplary “home” screen shot in accordance with oneembodiment of the present invention;

FIGS. 16 and 17 provide exemplary screen shots of various search resultsin accordance with various embodiments of the present invention;

FIG. 18 provides an exemplary screen shot, illustrating how message datamay be presented in accordance with one embodiment of the presentinvention;

FIG. 19 provides an exemplary screen shot of search results inaccordance with one embodiment of the present invention;

FIGS. 20 and 21 provide exemplary screen shots, illustrating how messagedata may be presented in accordance with various embodiments of thepresent invention; and

FIG. 22 illustrates a method in accordance with one embodiment of thepresent invention that uses at least one search term to identify resultsfrom different applications.

DETAILED DESCRIPTION

The present invention is described as a system and method for processing(receiving, analyzing, organizing, presenting, etc.) data stored on amobile phone. It should be appreciated, however, that the presentinvention is not limited to the processing of data for any particularpurpose, such as litigation (e.g., responding to discovery requests,etc.). It may also be used as an investigative tool (e.g., by humanresources, law enforcement, etc.), a compliance tool, or for personalreasons.

For example, a user generally goes through several mobile phones duringtheir life and may need to review data stored on a device to ensure thatdata of interest is not lost. The present invention can be used as atool to easily navigate through data stored on a computing device,regardless of the user's objective or the interest at issue. By way ofanother example, the present invention could be used in an officialcapacity, e.g., by law enforcement, to investigate a user's actions andactivities (e.g., during a particular time, involving a particularindividual, regarding a particular offense, etc.).

It should also be appreciated that the present invention is not limitedto data stored on mobile phones. While mobile phones are used herein toillustrate the present invention, the technology is equally applicableto other mobile (and non-mobile) devices, such as tablets, laptops, etc.This is because, it is not the device that gives rise to the presentinvention, but the data that is stored thereon. In other words, thepresent invention is instrumental for processing the types of data thatare generally stored on mobile devices, such as text messages, socialmedia data, etc. However, as technology advances, it is not uncommon forsuch data to also be stored on other devices, such as tablets, PCs, etc.For this reason, the present invention is not limited to mobile phones,and is equally applicable to other communication devices, both mobileand non-mobile.

A detailed description of the present invention will now be provided.During this discussion, it should be appreciated that like elementnumerals are used to describe like elements illustrated in one or morefigures.

As shown in FIG. 1 , preferred embodiments of the present inventionoperate in accordance with a Web host 102 in communication with a mobilecommunication device 108, such as a mobile phone, via a wide areanetwork (WAN) 100. In one embodiment of the present invention, a localcomputer 106, such as a personal computer (PC), is connected to themobile device 108 either directly or via the WAN 100. During thisconnection, the data is transferred from the mobile device 108 to thecomputer 106, either at the direction of the mobile device 108 (e.g.,uploaded) or the computer 106 (e.g., downloaded). The data is thenprovided to the Web host 102 via the WAN 100. This can be accomplishedeither directly, by providing the data directly to the Web host 102 viathe WAN 100, or indirectly, by providing the data to a networkedcomputer 104, such as a remote server, via the WAN 100. With respect tothe latter, the data would then be provided to the Web host 102 (e.g.,via the WAN, etc.), which again can be accomplished either at thedirection of the networked computer 104 (e.g., uploading) or the Webhost 102 (e.g., downloading).

While FIG. 1 depicts the preferred embodiment, it should be appreciatedthat other embodiments are within the spirit and scope of the presentinvention. For example, the networked computer 102 may be a backupdevice for the mobile device 108, such as Apple's iCloud™ Google Cloud™,or Verizon's Backup Assistant™, which may communicate (constantly,periodically, etc.) with the mobile device 108, and be in a position toprovide the Web host 102 with an exact copy of the data stored on themobile device 108. By way of another example, host software could beoperating on the local computer 106, which would allow the presentinvention to function offline (e.g., without a WAN, such as theInternet), on the local computer 106.

It should also be appreciated that the present invention is not limitedto the devices that are depicted in FIG. 1 and described herein. Forexample, the mobile communication device 108 may be any computing device(as discussed above), mobile or otherwise, the local computer 106, maybe a desktop, a laptop, or any other computing device, and the networkedcomputer 104 and/or the host computer 102 may include a plurality ofdevices (computers, servers, memory devices, etc.) operating in concert.By way of another example, the WAN 100 may be a local area network (LAN)or non-existent if host software is operating on the local computer 106.In this case, the communication channel between the local computer 106and the mobile device 108 can be any wired or wireless technologygenerally known to those skilled in the art. Thus, while preferred waysof receiving data have been discussed herein, the present invention isnot so limited.

It should also be appreciated that additional computers (e.g., networkedcomputers) (not shown in FIG. 1 ) are also within the spirit and scopeof the present invention. For example, once the data has been loadedonto (or accessible by) the Web host device 102, a reviewer may use adifferent, remote computer to log into the Web host application, selecta device (or data stored thereon), and navigate through the selecteddata (e.g., browse, search, etc.). Other reviewers may login via otherremote computers. Once the data has been annotated, results (e.g.,records identified as responsive, reports, etc.) can then be provided toan individual (e.g., third party) via another remote computer. Theresults can be provided in annotated form (e.g., making them easier toreview) or in raw form (e.g., ensuring that the data has not beenaltered).

In one embodiment of the present invention, as shown in FIG. 2 , thehost 102 may include at least one transceiver 212 (e.g., a networkconnection, etc.), at least one memory device 204 (e.g., hard drive,RAM, RAID, etc.), at least one processor 202, and at least oneapplication or program 208, which may or may not be stored in the memorydevice 204. The host 102 may also include at least one input device 210,such as a keyboard, mouse, etc., and at least one output device 206,such as a display. If the host 102 is providing a service to a pluralityof clients, the input/output would allow a staff member to access thesystem to provide updates, maintenance, customer support, etc. If thehost 102 is operated by a client (e.g., host software operating on alocal computer), the input/output would allow a reviewer to receive,review, and interact with the data (e.g., tag, run reports, etc.), asdiscussed in greater detail below.

It should be appreciated that the present invention is not limited toFIG. 2 , and a host that includes fewer, greater, or differentcomponents is within the spirit and scope of the present invention. Forexample, a memory that is external to the Web host device is within thespirit and scope of the present invention. It should also be appreciatedthat FIG. 2 is equally applicable to the networked computer, the localcomputer, and the mobile communication device, as depicted in FIG. 1 .For example, a local computer operating host software may include atleast one transceiver (e.g., network connection, USB, Bluetooth, NFC,etc.), at least one memory (e.g., RAM, ROM, etc.), a processor, at leastone application (e.g., a program for receiving and processing mobiledata), etc. Similarly, the mobile communication device may include atleast one transceiver (e.g., micro-USB, Bluetooth, NFC, cellulartransceiver, etc.), at least one memory, a processor, at least oneapplication (e.g., text message application, phone application, cameraapplication, social media application, etc.), etc. It should beappreciated, however, that while the local computer or the mobilecommunication device may include components that are similar (oridentical) to those depicted in FIG. 2 , the present invention is not solimited. For example, a mobile communication device that includes fewer,greater, or different components is within the spirit and scope of thepresent invention.

Regardless of whether the present invention is operating locally (e.g.,on a personal computer, etc.) or remotely (e.g., on a Web host, etc.),the client (referred to herein as the reviewer) will have an opportunityto navigate, interact with, and organize the data that was previouslystored on the mobile communication device. The present invention willnow be described using exemplary screen shots of what may be presentedto the reviewer. It should be appreciated that these screen shots aremerely exemplary and are not limitations of the present invention. Theymerely illustrate how the present invention may function. The presentinvention includes features that are not dependent on design (e.g.,shape, size, layout, etc.). Thus, a system that provides any of thefeatures discussed below, is within the spirit and scope of the presentinvention, regardless of how those features are presented to thereviewer.

In one embodiment of the present invention, as shown in FIG. 3A, aclient may have multiple cases loaded into (e.g., stored on) the system.See also FIG. 15 . For example, there may be a first case 302 a for apending litigation, a second case 302 b for an ongoing investigation,etc. Upon logging in, the client may be presented with a list of casesthat are associated with (e.g., available to) the client (e.g., 302 a,302 b, 302 c, 302 n) and allowed to select at least one case (e.g., 302c) from the list. The client may then be presented with a list ofdevices that are associated with that case (e.g., 304 a, 304 b). Forexample, a first device 304 a may be the plaintiffs mobile phone, thesecond device 304 b may be the defendant's PC, etc. The client may thenbe allowed to select at least one device (e.g., 304 a) from the list.The client may also be presented with different ways of navigating thedata, including browsing 306 a or searching 306 b. It should beappreciated that other navigational techniques are within the spirit andscope of the present invention. For example, a client may browse byother fields (e.g., by tag, reviewer, etc.). See, e.g., FIG. 15 .

Once a device has been selected, if selection is necessary, data storedon that device is presented to the client for review (hence, the term“reviewer”). This “presentation” can be done in a number of ways. In oneembodiment of the present invention, the system presents the reviewerwith a plurality of applications that are associated with the data. Thiscan be seen in FIG. 4A, where the mobile device includes data from aplurality of applications 402, including message (e.g., text messaging),phone, email, photo (e.g., camera), Instagram™, Facebook™, Twitter™, andWhatsApp™ applications. Identifying the applications that are on (oraccessible via) the mobile device can be done in a number of ways,including analyzing applications (or programs), folders, and/or filesthat are stored on the mobile device. In fact, many operating systemsare adapted to present the user with applications that are operating ontheir device in the “settings” menu. This same menu can be accessed bythe system to identify applications that are operating on the mobiledevice.

Once the categories (e.g., applications) are identified, the data (orindividual portions thereof) (e.g., files, etc.) can then be linked (orassociated) with at least one category. This can be done in a number ofways, including analyzing the folder where the file was stored (e.g., anemail file may be stored in an email folder, etc.) and/or analyzing thefile itself (e.g., file type, structure, content, etc.) (e.g., a .docfile designates a Word document, etc.). Once the individual files arelinked (or associated) with a particular application, the user can thenbrowse by selecting an application of interest.

For example, as shown in FIG. 4A, if the reviewer selects the “phone”category or application 402 a, they will be presented with phone data,which may include a phone log. The log may provide information on eachcall (or record), including, but not limited to, incoming, outgoing,answered, rejected, voicemail, voicemail content, name, number, date,time, duration, etc. This information may either be presented along witheach record or separately (e.g., in a pop-up window) if a particularrecord is selected (e.g., by clicking on a record, placing a pointerover a record, etc.). The records may be organized chronologically, orin some other fashion. For example, as shown in FIG. 4 , a first record404 a shows that “Mo” called on Jun. 18, 2019, at 10:26 PM, and thatcall was not answered. A second record 404 b shows that “Doug” called onJun. 18, 2019, at 6:55 PM, and that call was not answered. A thirdrecord 404 c shows that “Jonathan” called on Jun. 18, 2019, at 2:00 PM,and that call was answered. As discussed above, additional informationmay be provided (e.g., duration, was a voicemail left, what was thecontent of the voicemail, etc.).

When reviewing the data, the reviewer may be allowed to annotate eachrecord by selecting at least one “tag” from a plurality of tags, whereeach tag has a separate meaning. For example, a tag could show whether afile was reviewed, not reviewed, responsive to a discovery request,non-responsive, responsive to a particular topic (e.g., a particulardiscovery request, a particular investigative issue, etc.), privileged(e.g., attorney-client, work product, etc.), non-privileged, etc. Inaddition, or alternatively, tags can be automatically selected by thesystem (e.g., reviewed, not reviewed, presented for review, reviewer'sname, review date, review time, review device, etc.).

For example, as shown in FIGS. 13 and 14 , the system should be able tocontrol (and identify) who has access to the system and data storedthereon (or accessible therefrom). This may be by accomplished byrequiring an individual (e.g., during setup, login, etc.) to provideidentifying information, such as name, company, phone number, address,security information (e.g., password, security question, etc.), etc.This information may be used by the system to identify the individual,and thereby monitor the reviewer's activity (e.g., tag files as reviewedby a particular reviewer, etc.), limit accessibility, etc. For example,certain individuals may only have access to a particular case, data froma particular device, data on particular topics, etc.

While tagging can be accomplished in a number of ways, FIG. 4Aillustrates one embodiment of the present invention, where searchresults are provided in a first column, and tags are provided in asecond column 408. For each record, the reviewer may be able to access apull-down menu 410 and select from a plurality of tags. For example, thefirst record 404 a has been identified as “Privileged” 406 a, the secondrecord 404 b has been identified as “None,” etc. It should beappreciated that the specific manner in which tags are selected (e.g.,selecting from a pull-down menu, etc.) is a matter of design choice, andis not a limitation of the present invention.

Once a tag(s) has been selected, it will then be associated with therecord (or collection thereof, see below) for which it was selected.This association can be stored in a number of ways. For example, themeta-data for the record can be modified to include the selected tag(s).This is not preferable, as it alters the data that was originally storedon the mobile device. Thus, a separate database should be created andstored in memory (see, e.g., FIG. 2 at 204) that links (or associates)the tag(s) to the record. As shown in FIG. 4B, this could beaccomplished via a database that includes at least one identifier for arecord (e.g., a file name 412 a, a file location 412 b, an associatedapplication 412 c, a file ID 412 d, etc.) and at least one related tag(e.g., topic 412 e, reviewed 412 f, reviewer 412 g, etc.). Again, thetag(s) could be those selected by the reviewer, or automaticallyselected by the system. This would allow a reviewer (or the otherauthorized individual) to subsequently browse, search, filter, and/orreport by tag(s).

The identifiers are used (either individually or collectively) toidentify a particular record. While each identifier (or certainidentifiers) could be created by the system, they are preferably thesame identifiers used by the mobile device, thereby leaving the dataunaltered or identical to the data stored on the mobile device. Forexample, the name 412 a should be the name of the file (or record) usedby the mobile device, the location 412 b (e.g., folder, etc.) should bethe location of the file on the mobile device, and the ID 412 c shouldbe the number(s) used by the mobile device to identify the file. Thisinformation may be identified by analyzing the file, extractinginformation from the file, and/or extracting information from the file'smetadata.

In certain instances, the ID alone can be used to distinguish one filefrom another (see, e.g., Message_1 vs. Message_2). If, however, thereare identical IDs, other identifiers (e.g., name, location, application,etc.) may be used, either alone or together with the ID, to uniquelyidentify a particular file. For example, while Photo_1 may have the sameID (1234) as Message_1, they have different names, locations, andapplications. Thus, name+location could be used to uniquely identifyeach file, application+ID could be used to uniquely identify each file,etc., all of which are within the spirit and scope of the presentinvention.

Some records may also have more than one ID (not shown in FIG. 4B). Forexample, a text message may have a message_1D and a thread_1D, where thelatter identifies a particular thread (or back-and-forth communicationsbetween at least two individuals) and the prior identifies the recordwithin the thread. Thus, while different text messages may have the samethread_1D, and different text messages may have the same message_1D, notwo messages (from the same application) have the samethread_1D+message_1D. Thread and Message IDs are preferably assigned toeach record by the mobile device or the application for which the recordis associated. Those skilled in the art will understand that regardlessof which identifier (or combination thereof) is used to uniquelyidentify each record, the unique identifier can then be used to link (orassociate) individual records with at least one tag.

As shown in FIG. 5 , if the reviewer selects the “messages” category orapplication 402 b, they will be presented with message data, which mayinclude individual threads having individual records or messages (502 a,502 b, 502 c). The reviewer should also be provided with information oneach record, including, but not limited to, the record's content (text,emojis, photos, video, etc.), to, from, date, time, read, unread, etc.This information may either be presented along with each record orseparately (e.g., in a pop-up window). While the records are preferablyorganized chronologically, other structures are within the spirit andscope of the present invention.

For example, the first record 502 a (identified as from Doug, to Mo andArdy) was sent on Jun. 24, 2019, at 9:24 AM, the second record 502 b(identified as from Ardy to Mo and Doug), was sent at 9:28 AM, and thethird record 502 c (identified as from Ardy to Mo and Doug), was sent at9:30 AM. By presenting a message in this fashion (i.e., showing messagesthat preceded and succeeded the message in the thread), the message ispresented in context and may make more sense to the reviewer. This mayinclude messages that have been deleted (see e.g., “D” at 506) by theuser of the device. This is because, while a user may “delete” amessage, which results in the message being “flagged” so that it is notdisplayed on the mobile device, the message may still exist in memoryand is therefore retrievable by the system. Such information (e.g.,deleted, file size, thread_ID, message_ID, etc.) may be extracted fromthe file and/or extracted from the file's metadata.

As with phone records, a reviewer may be allowed to select at least onetag from a plurality of tags for each message. However, at least one tagmay also be selected for the entire thread. For example, the wholeback-and-forth between Doug, Ardy, and Mo may be privileged. Thus,instead of marking each message (or record), the entire thread could betagged as privileged 406 a. However, if only certain messages (orrecords) are privileged, then those records could be tagged, leaving theentire thread untagged (or at least untagged with respect to the issueof “privilege”).

As shown in FIG. 6 , if the reviewer selects the “photos” category orapplication 402 c, they will be presented with photo data, which mayinclude individual photos or records (602 a, 602 b, 602 c, 602 d). Thereviewer should also be provided with information on each record,including, but not limited to, the record's content, size, location,date, time, whether the photo was edited, whether the photo was sent toanyone, and if so, to whom and when, etc. This information may either bepresented along with each record or separately (e.g., in a pop-upwindow). Again, while the records are preferably organizedchronologically, other structures are within the spirit and scope of thepresent invention.

FIG. 6 illustrates an alternate way that “tags” can be presented to areviewer, i.e., as a box 406 d that overlays (or is adjacent to) acorresponding record 602 d. The reviewer could designate a “tag” byselecting a box (e.g., by clicking on it, etc.) and selecting at leastone tag (e.g., using a pull-down menu, etc.).

As shown in FIG. 7 , if the reviewer selects a social media application,such as “WhatsApp” 402 d, they will be presented with data for thatapplication, which may include photos 702 a, messages 702 b, or otherdata created, received, or shared using the application. With eachrecord, the reviewer should be provided with information including, butnot limited to, the record's content, if it was received, from who, ifit was shared, to who, if it was created (or edited), how, when any ofthe foregoing events took place (date, time), etc. This information mayeither be presented along with each record or separately (e.g., in apop-up window). Again, while the records are preferably organizedchronologically, other structures are within the spirit and scope of thepresent invention. Like message data, social media data can be tagged,either individual (e.g., 706 a), or as a group or thread (e.g., 706).This is because social media data is similar to message data, wherethere are back-and-forth communications between at least two people.

As with messages, each record may include multiple IDs, allowing thesystem to identify the record and at least one thread from which itbelongs. Because IDs are preferably created by the mobile device, or anapplication operating on the mobile device, whether a record has one IDor multiple IDs, is dependent on the application at issue. Twitter mayhandle records differently than Facebook. It is up to the system torecognize the application at issue and organize data for thatapplication accordingly. Similarly, some application store data locally(on the mobile device) and others store data remotely (e.g., in acloud). Again, it is up to the system to recognize the application atissue and retrieve data for that application accordingly. If the data isstored on the mobile device, the data can be retrieved from the deviceitself. If, however, portions of the data are stored in a cloud,portions that are stored on the device (e.g., user credentials, such asuser name, password, etc.) may be retrieved from the device, andportions that are stored on the cloud may be retrieved from the cloud,where the latter (access to the cloud) may require the former (e.g.,acquiring user credentials).

As previously discussed, a reviewer may browse the data by selecting acategory or an application and scrolling through portions of the data(i.e., data for that category). Alternatively, or in addition, areviewer may search for a search term (e.g., name, subject, date, periodof time, etc.). Results may include records that match or are closelyrelated (e.g., similar, etc.) and may be performed globally (i.e., onall the data) or on individual categories or applications.

For example, as shown in FIG. 9 , if a reviewer enters the name “Ardy”in a search field 900, then the system may provide search results, whichmay be from one application or many applications. For example, by notselecting an application, all data stored on the mobile device may besearched. If, however, the reviewer only wants to see results from aparticular application, then they could select that application beforeentering the search term.

This can be seen, for example, in FIG. 16 , where the reviewer issearching the message application for the term “Taylor.” The system maythen present the reviewer with matching messages. The system may alsoidentify other applications that includes matching records (e.g.,including the number of matches, etc.). If no application is selected,the system may present results from multiple applications. See, e.g.,FIG. 17 . Regardless of how the results are displayed, if the reviewerselects a record, the system may present the reviewer with additionalinformation on the record. For example, as shown in FIG. 18 , if amessage is selected, the system may present the message content alongwith other information (e.g., date, etc.), including other records inthe thread (e.g., preceding and/or succeeding).

In one embodiment of the present invention, the system may include a“story telling” feature, where data related to a search result (orrecord) is presented to the reviewer in order to provide context. Forexample, if the record 902 b is responsive to a search term, then thesystem may provide other records related to that search term. This mayinclude records that precede and succeed the record in the sameapplication (i.e., 902 a, 902 c). Such records may be identified usingthe record's thread ID (identifying records that have the same threadID) and message ID (identifying records that have message IDs thatprecede and succeed the record's message ID). This may also includerecords from another application that are related to the search result.This may include records that match or are similar to the search term,from the same person to the same person, include similar subject matter,took place immediately before, after, within the same hour, within thesame day, etc.

This can be seen, for example, in FIG. 10 , where a search result forDec. 24, 2019, includes both a WhatsApp record for that day and the webbrowser's history for that day. This could be the result of searchingfor a term in the data (globally) or searching for a term in WhatsApp,with the “story telling” feature selected, indicating a desire for thesystem to identify records that are deemed related.

For example, during a criminal investigation, the messaging applicationon a suspect's mobile device could be searched for the victim's name.The “story telling” feature could be used to provide the reviewer withinformation on what the suspect was doing immediately before thecommunication, immediately after the communication, other communicationswith the suspect, etc. It may be that the suspect went to Google andsearched for “identity theft” immediately after their last communicationwith the victim. Executing this feature may involve more than justsearching different applications for the same data (e.g., same searchterm).

For example, assume that system is searching for “Ardy.” The system mayidentify text messages that are to or from Ardy (see, e.g., FIG. 5 ) andsocial media involving Ardy (see, e.g., FIG. 7 ). However, the systemmay also be configured to collect and organize information concerningArdy, which may be extracted from various applications and/or records.For example, as shown in FIG. 8 , a contact record on Ardy (802 a)(stored on the mobile device) may include his phone number (802 b), hisemail address (802 c), and his mailing address (802 d), and otherrecords stored on the mobile device may include other identifiers, suchas government issued IDs (802 e) (e.g., driver's license, socialsecurity number, etc.), financial information (802 f) (e.g., Visa,Paypal, Zelle, bank accounts, etc.), etc. While processing the datastored on the mobile device, the system may also learn that some ofthese identifiers (e.g., email, phone, etc.) are linked to user_name AAkbar (802 g) on WhatsApp, to user_name Camp Clown on Facebook, etc.This collection of information (i.e., database 800) can then be used toidentify related records. This can be seen in FIG. 9 , where a searchfor “Ardy” pulls up records concerning “Ardy” and records concerning “A.Akbar.”

By way of another example, a search for “Ardy” may result in a textmessage from Ardy at 9:00 PM on Sep. 9, 2012, to both Doug and Mo,concerning a bank robbery. The system may then present the text messageto the reviewer along with related records (e.g., side-by-side, etc.),where the related records are identified by searching the data (e.g.,globally, by application, etc.) for a second search term that isextracted from the first search term's search result (i.e., the textmessage). This could be for records that concern Doug and/or Mo, arefrom the same date/time (i.e., 9:00 PM on Sep. 9, 2012) (or shortlybefore or after), involve the same subject matter (e.g., bank robbery),etc.

Again, the system could be configured to do this automatically or at therequest of the reviewer (e.g., by selecting the “story telling”feature). This feature could be used to identify related records indifferent applications, including the web browser, which may include,but is not limited to, websites visited, date/time/duration of visited,website credentials (e.g., user name, password), website activity (e.g.,postings, purchases, downloads, etc.), searches, search results,cookies, etc.

When reviewing the data, it may be necessary to understand the number ofsearch results for a search term. For example, as shown in FIG. 11 , areviewer may want to search the data for files that have been tagged as“responsive.” A search for this tag may result in the system identifyingthe number of search results in each category or application. In FIG. 11, there are only four photos that were tagged “responsive.” Withmessages, however, there are 301 records that were tagged as“responsive.” By knowing the number of search results (e.g., percategory, etc.), it allows the reviewer to understand the nature/extentof the search results. This is extremely important when reviewing mobiledata, which may not have a logical beginning or end. A search, forexample, could result in reviewing data having a date that is older thansome, newer than others, leaving a reviewer wondering if there werefiles that have not yet been reviewed.

As shown in FIG. 12 , a reviewer could search for the tag “notreviewed,” which is a tag that may be generated automatically by thesystem based on whether a record has or has not been presented to thereviewer (e.g., displayed), read, tagged, included in a search result,etc. This would allow a reviewer to “tie up loose ends” by reviewingdata that was not responsive to any search or was not encountered whilebrowsing. This can very easily be the case if the categories arefolders. Anyone who has ever connected their mobile phone to a computerin order to search for a file will understand that files are not alwayseasy to locate. For example, some photos (ones taken by the user) may bein a first folder, whereas others (ones shared by others) may be in asecond folder. Thus, the feature illustrated in FIG. 12 can be used toensure that a thorough review of the data has been conducted, and thatall records have been reviewed, or as least presented to the reviewerfor review.

It should be appreciated that other tags may also be selectedautomatically by the system, including who reviewed the data, when thedata was reviewed, action that was taken during the review, etc. Othertags may be more substantive, such as comments, allowing the review toprovide comments on each record or on a compilation thereof. Thereviewer may also be allowed to annotate the data (e.g., highlight text,etc.). However, it is preferred that such annotations be storedseparately from the data, allowing the original data to remainunaltered. The annotations could be stored in a database (see, e.g.,FIG. 4B), and could be stored as instructions (e.g., highlight thephrase “where shall we meet”). During a subsequent viewing of therecord, the record could be presented in its original form (e.g., ifannotations are turned off) or as annotated (e.g., if annotations areturned on), where the record is re-annotated (based on the instructions)at the time it is being presented. In an alternate embodiment, a copy ofthe record as annotated could be stored in the database. During asubsequent viewing of the record, the original record could be presented(e.g., if annotations are turned off) or the duplicate (annotated)record could be presented (e.g., if annotations are turned on). Bycreating a duplicate record, the original record remains unaltered.

It should also be appreciated that tags can also be browsed. See, e.g.,FIG. 19 . For example, if a particular tag (e.g., privileged) isselected, records associated with that tag would be presented to thereviewer. As before, if a record is then selected, information on thatrecord may be provided to the reviewer. If the record is a message, thatmay include the text message itself, the date/time it was received, andother messages in the same thread. See, e.g., FIG. 20 . It shouldfurther be appreciated that collective groups of records can also bebrowsed (or searched). For example, as shown in FIG. 21 , a reviewer maybe allowed to browse threads from a message application. If a thread isselected, the messages (or records) from the thread may be presented tothe reviewer (e.g., in chronological order).

A method of identifying records that match (or are related) to a searchterm in accordance with one embodiment of the present invention is shownin FIG. 22 . Starting at step 2200, a search term is received at step2202. As discussed above, this could be anything of relevance, rangingfrom a name, to a date (or range), to a tag, or other data or metadatathat may be of interest. At step 2204, records are identified that matchor closely resemble (e.g., abbreviations, etc.) the search term. Ifrecords are identified at step 2206, then the search results arepresented to the reviewer at step 2208. If, however, records are notidentified at step 2206, then data associated with the search term isidentified at step 2212. For example, if the search term is a name of anindividual, then data associated with that individual may include a dateof birth, phone number, email address, physical address, governmentissued ID (e.g., social security number, driver's license number, etc.),financial information (banking information, etc.), etc. This data canthen be used to identify an alternate name (e.g., an alternate username, etc.), or in the case of a generic search term, an alternatesearch term (e.g., for a Visa number, a linked Mastercard number couldbe used; for a driver's license number, a linked Social Security Numbercould be used; etc.) at step 2212. Records that match the alternatesearch term can then be displayed at step 2214, ending the method atstep 2216.

It should be appreciated at that the present invention is not limited tothe method illustrated in FIG. 22 , and that methods that includeadditional, fewer, or different steps are within the spirit and scope ofthe present invention. For example, regardless of whether there aresearch results at step 2206, alternate search results could beidentified at step 2214 (e.g., if the system is so configured, if the“story telling” feature is turned on, etc.), and presented to thereviewer. By way of another example, the alternate data (itself) couldbe used to identify search results. For example, search results for Johncould also include search results for John's date of birth, emailaddress, phone number, etc. By way of another example, search resultsfor a first term could also include search results for a second term,where the first and second terms are synonyms, one is a root and theother further includes a prefix, further includes a suffix, is a pluralform of the root, etc.

Having thus described several embodiments of a system and method forreceiving, analyzing, and organizing data stored on a mobilecommunication device, and presenting that data in a user-friendlyfashion, it should be apparent to those skilled in the art that certainadvantages of the system and method have been achieved. It should alsobe appreciated that various modifications, adaptations, and alternativeembodiments thereof may be made within the scope and spirit of thepresent invention. The invention is solely defined by the followingclaims.

What is claimed is:
 1. A web host for analyzing, organizing, andpresenting data stored on a mobile communication device, comprising: aserver adapted to receive the data from the mobile communication device,the data comprising at least one of a first set of records for a firstapplication and a second set of records for a second application; atleast one memory device in communication with the server and adapted tostore the data; and at least one program operating on the web host, theat least one program being adapted to: receive a first search term;search the first set of records for the first search term to identify athread comprising at least one matching record, the at least onematching record having a record identifier and a thread identifier; usethe record identifier and the thread identifier to identify a precedingrecord in the thread and a succeeding record in the thread; search thesecond set of records to identify at least one other record that isassociated with the first search term; and generate a webpage, thewebpage including at least the preceding record, the at least onematching record, and the succeeding record, in that order, and the otherrecord.
 2. The web host of claim 1, wherein the first application allowsa user of the mobile communication device to communicate with at leastone other user via at least one other mobile communication device. 3.The web host of claim 1, wherein the at least one memory device isfurther adapted to store at least one tag, wherein the at least one tagis selected from a plurality of tags available to a reviewer of the dataand linked to individual portions of the data.
 4. The web host of claim3, wherein at least one of the plurality of tags can be selected for theat least one matching record, the preceding record, the succeedingrecord, the at least one other record, and the thread, and whereinselection of the at least one tag for a corresponding record results inthe at least one tag being linked to the corresponding record in the atleast one memory device, and selection of the at least one tag for thethread results in the at least one tag being linked to the thread in theat least one memory device.
 5. The web host of claim 1, wherein theserver is adapted to receive the data from the mobile communicationdevice via at least one intermediate device connected to a wide areanetwork (WAN).
 6. The web host of claim 1, wherein the first search termcomprises a name of an individual, and the matching record comprises atext message that comprises at least one of (i) to the individual, (ii)from the individual, and (iii) mentions the individual.
 7. The web hostof claim 1, wherein the step of searching the second set of recordscomprises searching the second set of records for the first search termto identify at least one other matching record.
 8. The web host of claim1, wherein the step of searching the second set of records comprises (i)using a link that is common to both the first search term and a secondsearch term to identify the second search term, and (ii) searching thesecond set of records for the second search term to identify the atleast one other record, the second search term matching the at least oneother record.
 9. The web host of claim 8, wherein the link comprises atleast one of a user name, a phone number, an email address, an address,a government issued identifier, and financial information.
 10. The webhost of claim 1, wherein the search term comprises a temporal range, andthe record comprises a text message that was at least one of sent andreceived during the temporal range.
 11. A method for analyzing,organizing, and presenting data stored on a mobile communication device,comprising: receiving the data from the mobile communication device viaa wide area network (WAN), the data comprising at least one of a firstset of records for a messaging application and a second set of recordsfor a second application; storing the data in at least one memorydevice; making the data available to a reviewer; receiving a firstsearch term from the reviewer; searching the first set of records forthe first search term to identify a thread comprising at least onematching record, the at least one matching record having a recordidentifier and a thread identifier; using the record identifier and thethread identifier to identify a preceding record in the thread and asucceeding record in the thread; searching the second set of records toidentify at least one other record that is associated with the firstsearch term; and sending a webpage to the reviewer, the webpageincluding at least the preceding record, the at least one matchingrecord, and the succeeding record, in that order, and the at least oneother record, thereby providing the reviewer with context for the atleast one matching record.
 12. The method of claim 11, furthercomprising the step of storing at least one tag in the at least onememory device, wherein the at least one tag is selected from a pluralityof tags available to the reviewer linked to individual portions of thedata, and wherein at least one of the plurality of tags can be selectedfor the at least one matching record, the preceding record, thesucceeding record, the at least one other record, and the thread. 13.The method of claim 12, wherein selection of the at least one tag for acorresponding record results in the at least one tag being linked to thecorresponding record in the at least one memory device, and selection ofthe at least one tag for the thread results in the at least one tagbeing linked to the thread in the at least one memory device.
 14. Themethod of claim 11, wherein the step of receiving the data comprisesreceiving the data from the mobile communication device via at least oneintermediate device connected to the WAN.
 15. The method of claim 11,wherein the step of searching the second set of records comprisessearching the second set of records for the first search term toidentify at least one other matching record.
 16. The method of claim 11,wherein the step of searching the second set of records comprisessearching the second set of records for a second search term to identifyat least one other matching record, the second search term beingassociated with a link that is also associated with the first searchterm, wherein the link comprises at least one of a user name, a phonenumber, an email address, an address, a government issued identifier,and financial information.
 17. The method of claim 11, wherein the stepof receiving a first search term comprises receiving a temporal range,the record being a text message that was sent or received during thetemporal range.
 18. A system for reviewing data stored on acommunication device, comprising: a server adapted to receive the data,the data comprising a first set of records for a messaging applicationand a second set of records for a second application; a memory adaptedto store the data; and at least one program adapted to: receive a firstsearch term from a data reviewer; search the first set of records forthe first search term to identify a thread comprising at least onematching record, the at least one matching record having a recordidentifier and a thread identifier; use the record identifier and thethread identifier to identify a preceding record in the thread and asucceeding record in the thread; search the second set of records toidentify at least one other record that is associated with the firstsearch term; and display to the reviewer at least the preceding record,the at least one matching record, the succeeding record, and the otherrecord, at the same time.
 19. The system of claim 18, wherein the atleast one program is further adapted to identify a second search termthat is an alternative to the first search term.
 20. The system of claim19, wherein the step of searching the second set of records comprisessearching the second set of records for the second search term toidentify at least one other matching record.